Video streams captured by video cameras may suffer from a problem known as camera shake. For example, a video camera mounted at an area for, e.g., monitoring the area may from time to time experience shakes or vibrations due to shifting environmental factors, such as wind, vehicles (e.g. airplanes, trains, trucks, etc.) entering or leaving the area, construction work, etc. Such shakes or vibrations will induce movement of the video camera, thereby causing a captured video stream to shake.
Prior art systems for image stabilization reducing the effects of camera shake often include electronic manipulation of a video stream captured by the video camera before displaying the video stream. By doing so the viewer is presented with a stable view of the video stream despite the induced movement of the video camera. This electronic manipulation is based on determining an image shift between image frames of the video stream captured by the video camera. Depending on the image read out technique used, the image shift and hence the method of electronic manipulation of a video stream may differ. In case a global shutter is used, i.e. an entire image frame is read out simultaneously, the whole content of an image frame is shifted in order to compensate for the vibration of the video camera. In case a rolling shutter is used, i.e. an image is read out one or a few row(s) at a time, each row or a few rows in the image frames is/are shifted individually since they are exposed at different points in time. For the sake of simplicity, in the examples disclosed below, global shutter read out is used for explaining the electronic image stabilization, however, it should be understood that rolling shutter read out may as well be used for achieving the same result. Hence, the term image shift should be understood as a shift between the whole image content of two image frames or a shift between the image content of one or a few rows of two image frames.
An image shift determination may be made in various ways. One way of determining the image shift is to use sensors within the video camera to measure movement of the camera and with knowledge of the focal length of the camera lens, a corresponding image shift on a camera sensor may be calculated. Alternatively, features within the scene depicted by the video stream believed to be stationary between image frames may be tracked using image processing techniques.
Once the image shift has been measured each of a plurality of image frames of the video stream is shifted in accordance with the measured image shift to compensate for the image shift caused by camera movement. Thereby a viewer may be provided with a stable displayed view of the scene depicted by the video stream.
A prior art image stabilization system will be explained in more detail in connection with FIG. 1.
In FIG. 1(A) an example of a displayed video stream affected by video camera shake or vibration is presented. More precisely, a plurality of superimposed image frames of the video stream depicting a scene of a person 1 and a tree 2 is presented. The video camera shake or vibration induces movement of the video camera. Due to the movement of the video camera the person 1 and the tree 2 move to various positions within the displayed image frames. The superposition of image frames representative of the video stream of the scene captured at different positions of the video camera has an effect of obscuring visual information of the scene. Therefore, FIG. 1(A) presents a representation of an unstable video stream of the person 1 and the tree 2.
In FIG. 1(B) the image frames of the video stream are displayed after determining and applying image shift between the image frames. Hence, FIG. 1(B) is a representation of a video stream after electronic image stabilization according to the above has been applied to the unstable video stream presented in FIG. 1(A). In FIG. 1(B), the stabilized video stream is now centrally located and stable within the display. Hence, the image stabilization system operates to shift each image frame such that the person 1 and the tree 2 appear to be stable in the display. However, as the electronic image stabilization operates to shift entire image frames, part of the displayed video stream will not contain any displayed information and will therefore appear blank. Furthermore, as indicated in FIG. 1(B) by solid lines 3, 4, 5, 6, 7, which are representative of the edges of the displayed image frames, the edges of the video stream will continuously be moving as the image stabilization system operates to present a stable video stream of the scene. In practice, the image stabilization system, hereinbefore described, suffers a disadvantage in that although the image stabilization system operates to stabilize an image within the centre of a display, the peripheral region of the displayed video stream will be continuously moving, which is distracting to the viewer.
To alleviate this problem the prior art image stabilization systems operate to expand or zoom in the displayed view such that only the stable central part of the video stream is displayed. FIG. 1(C) presents the stabilized video stream depicting the scene of the person 1 and the tree 2, expanded to fill the display. In practice, the prior art image stabilization includes an always present digital zooming in of 10-20%. This implies that even when the image is stable, image information around the edges of the image frames is lost. Hence, peripheral image information will be lost, thereby greatly reducing the value of the image information of the video stream. The digital zooming in also implies that the video stream is displayed with a lower resolution than supplied by an image sensor of the video camera.